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DETAILED ACTION 

1 . This Office Action is in response to an AMENDMENT entered November 14, 
2005 for the patent application 09801795 filed on March 9, 2001 . 

2. The First Office Action of January 27, 2005 is fully incorporated into this Final 
Office Action by reference. 

Status of Claims 

3. Claims 1-24 are pending in this application. 

Claim Rejections - New Matter 

4. Claims rejected under 35 U.S.C. 1 12, first paragraph, as failing to comply with 
the written description requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to reasonably convey to one skilled in 
the relevant art that the inventor(s), at the time the application was filed, had possession 
of the claimed invention. Claims 5, 8, 9, 12, 18, 22, 23 and 24 cite the limitation of a 
"knowledge base" where the specification and the drawings are silent on such limitation. 

5. Claims rejected under 35 U.S.C. 1 12, first paragraph, as failing to comply with 
the written description requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to reasonably convey to one skilled in 
the relevant art that the inventor(s), at the time the application was filed, had possession 
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of the claimed invention. Claim 24 cites the following limitation that was not found in 

the specification and drawings: 

wherein the creating of the plan by the navigation planner comprises the 
navigation planner dynamically analyzing a current screen to determine a current 
state, determining a desired state associated with the problem, and dynamically 
identifying a sequence of the actions from the current state needed to achieve 
the desired state. 



Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the Invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed In the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

7. Claims 1-28 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Okerlund et al (U.S. Patent 6.690.371. referred to as Okerlund). 

Claim 1 

Okerlund anticipates a processor (Okerlund, c 3, 1 10-21); a computer memory 
coupled to said processor( Okerlund, c 3. 1 10-21); and a screen fingerprinter stored in 
said computer memory, wherein said fingerprinter comprises a decision tree that selects 
at least one region and/or pattern of screens of a presentation space of a computer 
application to be captured such that an occurrence of the said at least one region and/or 
pattern enables the decision tree is uniquely identify each of the screens (Okerlund. c 
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3, 1 22-35; c 8, 28-42; Examiner's Note (EN): H 12 applies; a fingerprinter is simply the 
software that acquires specific screen presentation data related to at least one 
computer; decision tree is synonymous with hierarchical data structure in the form of a 
binary tree which has a decision point at each node that is not a leaf node). 
Claim 2 

Okerlund anticipates said fingerprinter allows a user to modify which portion of a 
screen comprises said region and/or pattern and which attributes of said region and/or 
pattern to examine (Okerlund, c 3, 1 35-47). 
Claim 3 

Okerlund anticipates said fingerprinter creates the decision tree based on said at 
least on region and/or pattern such that after each screen is compared to the region or 
pattern at each decision node, a screen identifier will come to a different end node of 
said decision tree for each screen (Okerlund, c 8, 1 28-42). 
Claim 4 

Okerlund anticipates said fingerprinter allows a user to modify said decision tree 
by modifying the comparisons at the decision nodes (Okerlund, c 3, 1 35-47; c 8, 1 27- 
50). 

Claim 5 

Okerlund anticipates a processor (Okerlund, c 3, 1 10-21); a computer memory 
coupled to said processor (Okerlund, c 3, 1 10-21); a user interface and a state recorder 
stored in said computer memory, wherein said state recorder records in said computer 
memory a knowledge base which comprises each screen of the presentation layer of a 
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computer application, the keystrokes and/or programs necessary to reach each state, 
the available actions from each state of each screen and the effect of any actions 
available each state through navigating said computer application in said user interface 
(Okerlund, c 3, 1 10-47, c 4, 1 20-40; c 5, 1 20-23; EN: H 12 applies; a knowledge base 
without enablement is a memory limitation). 
Claim 6 

Okerlund anticipates said processor generates said file while a user navigates 
said another computer system in said user interface (Okerlund, c 4, 1 37-42). 
Claim 7 

Okerlund anticipates said processor generates said file while automatically 
navigating said another computer system (Okerlund, c 4, 1 29-36). 
Claim 8 

Okerlund anticipates a fingerprint of each screen is included in said knowledge 
base (Okerlund, c 4, 1 20-40; EN: such would happen as each screen is remotely 
viewed; EN: a knowledge base without enablement is a memory limitation). 
Claim 9 

Okerlund anticipates pre-conditions and post-conditions for each state are 
included in said knowledge base (Okerlund, c 4, 1 54-63; EN: a knowledge base without 
enablement is a memory limitation). 
Claim 10 

Okerlund anticipates a processor (Okerlund, c 3, 1 10-21); a computer memory 
coupled to said processor (Okerlund, c 3, 1 10-21); at least one computer application 
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model stored in said computer memory (Okerlund, c 3, 1 10-21); and a navigation 
planner stored in said computer memory (Okerlund, c 8, 1 29-42; EN: a hierarchical 
data stnjcture is a form of a navigation planner); wherein when said navigation planner 
receives a problem statement, said navigation planner accesses said at least one 
computer application model to create a plan of solving said problem statement and 
executes said plan (Okerlund, c 8, 1 43-50; EN: H 12 applies; problem statement is a 
change made to a slice and the execution follows a binary tree). 
Claim 11 

Okerlund anticipates when said plan fails, said navigation planner creates a new 
and different plan to solve said problem statement (Okerlund, c 8, 1 60-67; EN: a new or 
different plan results from the addition or removal of a slice wherein the hierarchical 
data structure is changed). 
Claim 12 

Okerlund anticipates a processor (Okerlund, c 3, 1 10-21); a fingerprinter 
(Okerlund, c 3, 1 10-21; EN: a fingerprinter is simply the software that acquires specific 
screen presentation data related to at least one computer ); a recorder (Okerlund, c 3, 1 
10-21 ; EN: such is memory); and a user interface (Okerlund, c 3, 1 36-47); wherein said 
fingerprinter selects at least one region and/or pattern of the screens of the presentation 
space of a computer application to be captured such that said at least one region and/or 
pattern of each screen is unique (Okerlund, c 3, 1 22-35; EN: the extraction of medical 
image data will be unique); wherein said state recorder records in said computer 
memory a knowledge base which comprises each screen of the presentation layer of a 
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computer application, tlie keystrokes and/or programs necessary to reach each screen, 
a fingerprint of each screen, the available actions from each screen and the effect of 
any actions available in each screen through navigating said computer application in 
said user interface (Okerlund, c 3, 1 10-47, c 4, 1 20-40; ; EN: a knowledge base without 
enablement is a memory limitation); wherein additional relationships between said 
screen and said knowledge base can be input through said user interface such that said 
computer application model generator can model said computer application (Okerlund, 
c 3, 1 36-47; EN: H 12 applies; a knowledge base without enablement is a memory 
limitation). 
Claim 13 

Okerlund anticipates a processor (Okerlund, c 3, 1 10-21); a computer memory 
(Okerlund, c 3, 1 10-21); a runtime agent stored in said computer memory (Okerlund, c 
3, 1 10-21 ; EN: run-time agent is an object (software) that is involved in the execution of 
the software and would of consequence be stored in the computer memory); and at 
least one computer application model stored in said computer memory, said model 
modeling at least one computer application (Okerlund, c 3, 1 10-21 ; EN: such is the 
operation of a computer); wherein when said processor receives a problem statement, 
said runtime agent accesses said at least one computer application model to 
intelligently reason out a goal-oriented plan and accesses the modeled computer 
applications to execute the tasks necessary to solve said problem statement (Okerlund, 
c 8, 1 60-67; EN: H 12. applies; such is computer science and the addition or removal of 
a slice). 
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Claim 14 

Okerlund anticipates taking a screen capture of each screen of the presentation 
layer of a computer application (Okerlund, c 8, 1 27-67; EN: such are the slices) 
selecting areas of said screen captures to be examined for the presence of an attribute 
in said area (Okerlund, c 8, 1 7-17; EN: such are ray definitions for every pixel) ; and 
creating a decision tree such that each of said screen captures has a unique end node 
of said decision tree (Okerlund. c 8, 1 28-42). 
Claim 15 

Okerlund anticipates said areas (claim 14) are selected automatically (Okerlund, 
c 8, 17-17). 
Claim 16 

Okerlund anticipates said areas (claim 14) are selected manually (Okerlund, c 3, 
I 36-47; c 8, 17-17). 
Claim 17 

Okerlund anticipates said decision tree is created manually (Okerlund, c 3, 1 36- 
47; c 8, 1 27-47; EN: manual adjustments are made by the technologist which then affect 
the creation of the decision tree). 
Claim 18 

Okerlund anticipates accessing said computer application (Okerlund, c 3, 1 10- 
21); navigating said computer application (Okerlund, c 8, 1 27-47); and recording in a 
knowledge base each screen of the presentation layer of said computer application, 
keystrokes and/or programs necessary to reach each state of each screen of said 
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computer application, the states of each screen, and the effect of any actions taken on 
each screen (Okerlund, c 3, 1 10-21; c 8, 1 17-67; EN: H 12 applies; such is a typical 
computer operation; ; a l^nowledge base without enablement is a memory limitation). 
Claim 19 

Okerlund anticipates said computer application is navigated automatically 
(Okerlund, c 3, 1 10-21; EN: computers run programs automatically; navigating from 
one program step to the next). 
Claim 20 

Okerlund anticipates said computer application is navigated manually (Okerlund, 
c 3, 1 10-21; EN: computers require manual inputs to run automatically). 
Claim 21 

Okerlund anticipates receiving a problem statement at a computer system 
(Okerlund, c 3, 1 10-21); accessing at least one computer application model that 
encapsulates information on how at least one computer application is controlled and/or 
data is accessed (Okerlund, c 3, 10-21; c 8, 1 17-67); planning a path through said at 
least one computer application that will achieve the goal of said problem statement 
(Okerlund, c 3, 1 10-21; c 8, 1 17-67); and executing said path (Okerlund, c 3, 1 10-21; c 
8, 1 17-67; EN: 11 12 applies). 
Claim 22 

Okerlund anticipates taking a screen capture of each screen of the presentation 
layer of a computer application (Okerlund, c 8, 21-27); selecting areas of said screen 
captures to be examined for the presence of an attribute in said area (Okerlund, c 8, 1 
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7-17); creating a decision tree such that each of said screen captures has a unique end 
node of said decision tree (Okerlund, c 8, 27-42); accessing said computer application 
(Okerlund, c 3, 1 10-21); navigating said computer application (Okerlund, c 3, 1 10-21; 
EN: same as running a computer application); and recording in a knowledge base each 
screen of the presentation layer of said computer application, keystrokes and/or 
programs necessary to reach each state of each screen of said computer application, 
the states of each screen, and the effect of any actions taken on each screen 
(Okerlund. c 3, 1 10-21; c 8, 1 17-67; EN: H 12 applies; such is a typical computer 
operation; a knowledge base without enablement is a memory limitation). 
Claim 23 

Okerlund anticipates allowing a user to insert additional relationships and 
commands into said knowledge base (Okerlund, c 3, 36-47), 
Claim 24 

Okerlund anticipates a processor (Okerlund, c 3, 1 10-21); a computer memory 
coupled to said processor( Okerlund, c 3. 1 10-21); and a screen fingerprinter stored in 
said computer memory, wherein said fingerprinter comprises a decision tree that selects 
at least one region and/or pattern of screens of a presentation space of a computer 
application to be captured such that an occurrence of the said at least one region and/or 
pattern enables the decision tree is uniquely identify each of the screens (Okerlund, c 
3, 1 22-35; c 8, 28-42; EN: a fingerprinter is simply the software that acquires specific 
screen presentation data related to at least one computer; decision tree is synonymous 
with hierarchical data structure in the form of a binary tree which has a decision point at 
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each node that is not a leaf node); a recorder stored in the computer memory, the 
recorder recording a knowledge base which comprises the plurality of screen captures, 
one or more inputs and/or programs necessary to reach the application states indicated 
by each of the screen captures, one or more actions available from each of the states, 
and the effects of undertaking each of the actions available from each of the states 
(Okerlund, c 3, 1 10-47; c 8, 1 7-42; EN: a knowledge base without enablement is a 
memory limitation); and a navigation planner that receives a problem statement, creates 
a plan of solving the problem statement by using the knowledge base, and executes the 
plan (Okerlund, c 8, 1 43-50; EN: problem statement is a change made to a slice and 
the execution follows a binary tree), wherein the creating of the plan by the navigation 
planner comprises the navigation planner dynamically analyzing a current screen to 
determine a current state, determining a desired state associated with the problem, and 
dynamically identifying a sequence of the actions from the current state needed to 
achieve the desired state (Okerlund, c 8, 1 43-50; EN: H 12 applies; problem statement 
is a change made to a slice and the execution follows a binary tree; binary tree has 
decision points (nodes or states) branching to leaves; computer programs or 
applications are dynamic as such moves through execution; binary tree identifies the 
path to reach the desired state or node). 
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Response to Arguments 

8. Applicant's arguments filed on November 14, 2005 related to Claims 1-24 have 
been fully considered but are not persuasive. 



In reference to Applicant's argument: 

Applicant traverses the characterization of Okerlund as relevant prior art, given that Okerlund does not 
teach or suggest the present invention. Specifically, as noted in the abstract, Okerlund is directed to: 

A method for rapid extraction and visualization of relevant data from a volume of image data 
includes rapidly producing reduced-fidelity images derived from an image volume, the reduced 
fidelity images having an adjustable visual parameter; adjusting the visual parameter of the 
reduced-fidelity images during the rapid production to select a desired adjustment; producing a 
full-fidelity image derived from the image volume; and applying the selected adjustment to the full- 
fidelity image. 

Thus, Okerlund teaches the visualization of image data, not the capture of an screen image and 
processing of the captured screen image to control the application which generated the image, indeed, no 
where does Okerlund describe or suggest the capturing of any information displayed on a screen and 
perform decision analysis based on this information capture as provided by the present invention, as 
explained below, jflf 56-58. 

Examiner's response: 

nil. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. First Office Action applies. 



In reference to Applicant's argument: 

The present invention, is directed to the automated control and execution of programs by analyzing the 
inputs used to generate certain program states, which are ascertained by polling a screen capture of a 
running application program. The present invention seeks to address the problem of integrating legacy 
and newer systems fl 6; TI 7. This is accomplished with runtime agents that use models to intelligently 
navigate systems, so that a single user interface may be used to control multiple applications. TJ 18. To 
achieve this, the runtime agents determine the state of an application by polling the screen displayed by 
the application. H 12. Through this polling, unique identifiers may be found and noted, thereby informing 
the integrating program of the state of the program, jl 12. Alternatively, a programmer may select regions 
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of a screen for the runtime agents. ^ 13. The present invention provides for models that are used by 
runtime agents to intelligently navigate systems. Embodiments of the present invention may employ a 
recorder that "records in the computer memory a plan domain file which comprises each state of the 
presentation layer of another computer system, the available actions from each state, and the effect of 
any actions available in each state though navigating the other computer system in the user interface." ^ 
20. Embodiments of the present invention may employ a "fingerprinter" that "captures sections of each 
screen of the presentation space from at least one other computer system." 28. No where in Okerlund is 
there any suggestion of a runtime agent acquiring a screen capture of a presentation space. Moreover, 
Okerlund does not teach or suggest the simultaneous use of two different processes — one being the 
runtime agent and the other being the application from which the runtime agent captures sections of each 
screen of the presentation space. 

Exanniner's response: 

^11. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. First Office Action applies. 



In reference to Applicant's argument: 

The system of claim 1 comprises a "screen fingerprinter" stored in a computer memory that "selects at 
least one region and/or pattern of the screens of the presentation space of a computer application to be 
captured such that said at least one region and/or pattern of each screen is unique." Claim 1 (emphasis 
added). Similarly, the specification notes that a fingerprinter "captures sections of each screen of the 
presentation space from at least one other computer system." H 28. Nowhere does Okerlund teach or 
suggest the capturing of a region and/or pattern of a screen of a presentation space. Examiner's 
statement that a "a fingerprinter is simply the software that acquires specific screen presentation data 
related to at least one computer" is an oversimplification of the meaning of "screen fingerprinter" as 
understood by one of ordinary skill in the art. The fingerprinter, as noted in the claim language and 
specification referenced above, acquires its data from data that is actually displayed upon a screen of a 
presentation space, not merely data that might eventually be displayed or modified to be displayed, as is 
Okerlund. In Okerlund, the end result and objective is to render some visualization on a screen. The 
present invention, in contrast, uses what is already being displayed on a screen by another computer 
application as a data source for runtime agents. This understanding is confirmed by the specification, 
which notes that a "fingerprinter in the computer memory captures sections of each screen of the 
presentation space from at least one other computer system." U 28. Thus, the fingerprinter's source is the 
actual screen display, which in Okerlund, the image on the screen is never used as a source of 
information. Moreover. Okerlund does not disclose the use of one process (performed by "a screen 
fingerprinter") to observe the operation of another process ("a computer application"), by monitoring the 
other process* state by selecting "at least one region and/or pattern of the screens of the presentation 
space of the other process. 



Examiner's response: 
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I1 1 1 . below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. Okerlund's fingerprinter is fully described @ c3:25-30. First 
Office Action applies. 



In reference to Applicants argument: 

Examiner cites col. 2, 1 1 . 22-35 of Okerlund. Here, Okerlund merely describes a system for displaying 
data acquired from an exam prescription subsystem, which acquires its data from a medical scan. 
Okerlund in no way teaches or suggests the use of a screen capture of any Image displayed as part of an 
image space. The present Invention comprises "a screen finger printer" which selects obtains its data 
from images presented on a screen, see claim 1, Okerlund, in contrast, obtains its data from a medical 
scan. Col. 3, 1 . 25-46. Okerlund does in any way teach the data acquisition of images displayed on a 
screen, and indeed never teaches the use of one process to observe and select the display of another 
process. 

Examiner's response: 

1 1 . below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. Okerlund's fingerprinter is described @ c3:25-30 ... the 
acquired images are archived for future use ... a medical scan does not disqualify 
Okerlund ... multiple processes are not addressed in claim 1. First Office Action 
applies. 



In reference to Applicant's argument: 
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With respect to claim 2, Examiner cites col. 3,1 1 .35-47 of Okerlund. Here. Okerlund does not disclose or 
suggest the use of a fingerprinter, which is understood as an agent that selects a portion of an image 
already being displayed by another application. 

Examiner's response: 

H 1 1. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. Okerlund's fingerprinter is described @ c3:25-30 First 
Office Action applies. 

In reference to Applicant's argument: 

With respect to claim 3, Examiner cites col. 8, 1. 28-42. Here, Okerlund does not teach or discuss the use 
of a "decision tree." Although a decision tree is one type of hierarchical data structure used to make 
decisions. The data structure noted in Okerlund is merely of data, not data that permits the making of 
decisions. Claim 3 is not anticipated by Okerlund and should be allowed. 

Examiner's response: 

H 1 1 . below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. Simply, hierarchical data can be a binary tree ... computers 
make decisions in accessing appropriate data in the tree ... decision tree. First Office 
Action applies. 

In reference to Applicant's argument: 

With respect to claim 4, Examiner cites col. 3, 1. 35-47. For the reasons stated above, Okerlund neither 
teaches the use of a "fingerprinter" nor the use of a "decision tree." Claim 4 is not anticipated by Okerlund 
and should be allowed. 
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Examiner's response: 

nil. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. See above comments. First Office Action applies. 



In reference to Applicant's argument: 

Claim 5 has been amended to include "a user interface" in order to provide an antecedent basis for "said 
user interface." The system of claim 5, as amended, comprises "a recorder" that records in computer 
memory "a plan domain file which comprises each screen of the presentation layer of a computer 
application, the keystrokes and/or programs necessary to reach each state, the available actions from 
each state of each screen and the effect of any actions available in each state through navigating said 
computer application in said user interface." Examiner cites Col. 3, 1 1 . 10-47 and col. 4, 1 1 . 20-40. 
Neither here, nor elsewhere, does Okerlund teach or suggest the use of a "recorder," which by its plain 
terms indicates something which records a state (which is understood by one of ordinary skill, in light of 
the claims and specification, to be the recording of the state of another computer application). Nothing In 
Okerlund teaches or suggests the use of a domain file that comprises "each screen of the presentation 
layer of a computer application." Nothing in Okerlund teaches or suggests the use of a domain file that 
comprises "the keystrokes necessary to reach each state." Nothing in Okerlund teaches or suggests the 
use of a domain file that comprises "the available actions from each state of each screen and the effect of 
any actions available in each state through navigating said computer application in said user interface." 
Indeed, nothing is Okerlund teaches or suggests the use of an agent to record the operations of a 
computer application by recording what is displayed upon a screen. 

Examiner's response: 

H 1 1 . below applies. The claims and only the claims form the metes and bounds 

of the invention. The Examiner has full latitude to interpret each claim in the broadest 

reasonable sense. Limitations appearing in the specification but not recited in the claim 

are not read into the claim. Okerlund's fingerprinter is described @ c3:25-30 has 

storage or a recorder. Again, Okerlund's fingerprinter addresses "each screen of the 

presentation layer of a computer application ... simply Okerland has a computer ... 

application ... screens ....and Okerlund stores data. Typically, the computer display 

responds to keystrokes and since Okerlund stores the monitor image, Okerlaund stores 
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the keystrokes that established the image ... albeit in a somewhat different form ... but 
sufficient to anticipate the limitations of the subject claim. The available actions and 
effects from each screen ... keystrokes are part of the software operating system that is 
stored in the computer memory. Applicant is silent in claim 5 concerning the use of 
agents. First Office Action applies. 

In reference to Applicant's argument: 

With respect to claim 6. Examiner cites col. 4, 1 1 . 37-42. Here, Okerlund merely discloses the viewing of 
a 3-d visualization. Okerlund does not teach or suggest the generation of a file based upon an user's 
navigation of another computer system. The term "navigation." as understood by one of ordinary skill in 
the art in light of the claims and specification, refers to the interaction between a user and another 
computer application as a user explorers or undertakes use of the computer application. Okerlund merely 
discloses the remote viewing of what is already being displayed, not the recording of the activities of a 
user as the user interacts with another computer application. 

Examiner's response: 

^11. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. The Okerlund user is navigating the system ... albeit the 
remote system ... from processor generated files. First Office Action applies. 

In reference to Applicant's argument: 

With respect to claim 7. Examiner cites col. 4, 1 1 . 29-36. Here, Okerlund does not teach or suggest the 
automated navigation of another computer application. 

Examiner's response: 

I1 11 . below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
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are not read into the claim. Automated navigation is synonymous with a computer 
process wherein the computer will automatically execute a process involving various 
functions of the computer. Such a process is addressed by Okerlund @ c4. 1 29-36. 

In reference to Applicant's argument: 

With respect to claim 8, Examiner cites col. 4.1 .20-40. As noted before, Okerlund does not teach or 
suggest the use of fingerprints nor domain files. 

Examiner's response: 

nil below applies. The claims and only the claims form the metes and bounds 

of the invention. The Examiner has full latitude to interpret each claim in the broadest 

reasonable sense. Limitations appearing in the specification but not recited in the claim 

are not read into the claim. Okerlund archives images ... fingerprints. First Office 

Action applies. 

In reference to Applicant's argument: 

With respect to claim 9, Examiner cites col. 4, 1 1 , 54-63. Here. Okerlund does not include 
pre-conditions or post-conditions for each state. 

Examiner's response: 

nil. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim, Okerlund teaches filtered data classified into separate 
categories. Such data has records and each record has a pre and post ... such are the 
nature of records. First Office Action applies. 
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In reference to Applicant's argument: 

The system of claim 10 comprises "a computer application model." As understood by one of ordinary skill 
in the art in light of the specification and claims, "a computer generated model" is generated by a 
recorder, thus provides information concerning the various states of an application, including the 
fingerprints of a state, the keystrokes needed to obtain the state, and the effect of any actions available in 
each state through navigating said computer application. H 28; H 20. These are not disclosed in Okerlund, 
as cited by the Examiner. Col. 3, 1 10-21. 

Moreover, the system of claim 10 comprises "a navigation planner." Examiner states that a hierarchical 
data structure is a form of a navigation planner. This is incorrect. By its plain terms, a navigation planner 
plans how a computer application is to be navigated. A mere hierarchical data structure does not perform 
any actions on its own. An navigation planner, in contrast, must plan how a computer application is to be 
navigated to progress from one state to another. Nothing in Okerlund, as cited by the Examiner, Col. 8. 1. 
29-42, suggests a navigation planner, let alone the use of an agent which navigates another computer 
program. 

Examiner's response: 

If 1 1. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. Applicant appears to equate "a computer generated 
model" with "a computer application model." See comments related to claim 5 above. 
The inherent nature of a hierarchical data structure sets forth relationships on how a 
computer application will process or navigate through the hierarchy ... Okerlund's binary 
tree and leaf nodes manifest a process from one state (node) to another. First Office 
Action applies. 



In reference to Applicant's argument: 

With respect to claim 1 1 , Examiner cites col. 8, 1 1 . 60-67. Nothing in Okerlund suggests the use creation 
of a new navigation plan, that is, a new way by which a computer application is to be navigated or 
controlled. 



Examiner's response: 
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H 1 1. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. See the "EN" in the first office action @ page 5, lines 1-5). 
First Office Action applies. 

In reference to Applicant's argument: 

Claim 12 comprises a "recorder," which is synonymous with "recorder." As discussed above in connection 
with claim 5, Okerlund does not teach or suggest a recorder. Claim 12 also comprises a "fingerprinter." As 
discussed above in connection with claim 5, Okerlund does not teach or suggest a recorder. Claim 12 
also requires the ability to form additional relationships between a screen and a domain file can be input 
through said user interface such that said computer application model generator can model said computer 
application. Examiner cites col. 3, 1 1 . 36-47. Here, Okerlund does not teach or suggest the modeling of a 
computer application. Moreover, Okerlund does not teach or suggest the use of domain files, as 
discussed above in connection with claim 5. 

Examiner's response: 

nil. below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. It is axiomatic that "recorder" is synonymous with "recorder." 
See the comments related to claim 5 above. Neither the original claims nor the 
amended claims address the limitation to "form additional relationships between a 
screen and a domain file can be input through said user interface such that said 
computer application model generator can model said computer application." The 
amended claim 12 does not limit by the use of domain files. First Office Action applies. 



In reference to Applicant's argument: 
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Claim 13 comprises a "runtime agent." Examiner misconstrues the meaning of runtime agent as used in 
the application. A run-time agent: "use the models to intelligently navigate each system" and "can be used 
to implement systems with a single user interface." U 18. Okerlund does not teach or suggest the use of 
runtime agents, which are not merely "involved in the execution of code." but must be capable of 
navigating another computer application. Claim 13 further comprises a "computer application model" 
which, as described above in connection with claim 10, is not taught or suggested by Okerlund. 

Examiner's response: 

nil below applies. The claims and only the claims form the metes and bounds 

of the invention. The Examiner has full latitude to interpret each claim in the broadest 

reasonable sense. Limitations appearing in the specification but not recited in the claim 

are not read into the claim. See comments related to claim 10 above. First Office 

Action applies. 

In reference to Applicant's argument: 

Claim 14 comprises a step of "taking a screen capture of each screen of the presentation layer of a 
computer application." Examiner cites col. 8, 1 . 27-67. Here, Okerlund is discusses rendering of a three- 
dimensional Image based upon image slices obtain from a medical scan. Nothing In Okerlund teaches or 
suggests taking a "screen capture," that is, the capturing of an Image of what is displayed on a screen. 

Claim 14 further comprises "selecting areas of said screen captures to be examined for the presence of 
an attribute in said area." Examiner cites col. 8, 11. 7-17. Claim 14 further comprises "creating a decision 
tree such that each of said screen captures has a unique end node of said decision tree." 

Examiner's response: 

I1 1 1 . below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. The screen capture is Okerlund's segmentation process 
related to the rendering or display which is done for each screen or display related to 
presentation layer (slice) and is involver in a computer implementation (Fig. 5) ... net 
effect is a screen capture ... albeit capturing before the actual display. The analysis of 
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the slices is what it is all about in reviewing medical data. Okerlund teaches a binary 
tree and leaf nodes @ c 8, 1 28-42. First Office Action applies. 

In reference to Applicant's argument: 

With respect to claims 15 and 16, Examiner cites col. 3, 11. 36-47 and col. 8,11.7-17. Here, Okerlund 
does not teach or suggest the use of screen captures, and thus does not anticipate the requirement of 
claim 15 and claim 16 that areas within such screen captures are, respectively, selected automatically 
and manually. 

Examiner's response: 

U 1 1 . below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. See claim 14 discussion above. Manual and automatic 
processing was cited by Okerlund @ c3, 1 36-47; c 8. 1 7-17. First Office Action applies. 

In reference to Applicant's argument: 

With respect to claim 1 7, Examiner cites col. 3,11. 36-47; col. 8, 1 1 . 27-47. As discussed above in 
connection with claim 3, Okerlund does not teach or suggest the use of a decision tree. Claim 1 7 is not 
anticipated by Okerlund and should be allowed. 

Examiner's response: 

H 10. below applies. The claims and only the claims form the metes and 
bounds of the invention. The Examiner has full latitude to interpret each claim in the 
broadest reasonable sense. Limitations appearing in the specification but not recited in 
the claim are not read into the claim. See discussion related to claim 3 above. First 
Office Action applies. 



In reference to Applicant's argument: 
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Claim 18 requires the "recording in a plan domain file each screen of the presentation layer of said 
computer application, the keystrokes necessary to reach each state of each screen of said computer 
application, the states of each screen, and the effect of any actions taken on each screen." As discussed 
above with respect to claim 5, Okerlund does not teach or suggest any of these features. 

Examiner's response: 

I1 1 1 . below applies. The claims and only the claims form the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. Applicant is silent in claim 18, amended, regarding 
"recording in a plan domain file each screen of the presentation layer of said computer 
application, the keystrokes necessary to reach each state of each screen of said 
computer application, the states of each screen, and the effect of any actions taken on 
each screen." 

In reference to Applicant's argument: 

Claim 21 requires "accessing at least one computer application model that encapsulates information on 
how at least one computer application is controlled and/or data is accessed." As discussed above in 
connection with claim 10, Okerlund does not teach or suggest the use of a computer application model. 
Claim 21 further comprises a step of "planning a path through said at least one computer application that 
will achieve the goal of said problem statement." As discussed above in connection with claim 10, 
Okerlund does not teach or suggest the use of navigation planning. 

Examiner's response: 

nil. below applies. The claims and only the claims fonn the metes and bounds 
of the invention. The Examiner has full latitude to interpret each claim in the broadest 
reasonable sense. Limitations appearing in the specification but not recited in the claim 
are not read into the claim. See comments related to claims 5 and 10 above. Okerlund 
achieves the goal by rendering a sliding window model ... computer implementation ... 
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the route to the goal is a computer process path ... which is related to a problem 
statement. First Office Action applies. 



In reference to Applicant's argument: 

Claim 22 comprises a step of "taking a screen capture of each screen of the presentation layer of a 
computer application." As discussed above in connection with claim 1. Okerlund does not teach or 
suggest the use of a "screen capture." Claim 22 further comprises a step of "creating a decision tree such 
that each of said screen captures has a unique end node of said decision tree." As discussed above in 
connection with claim 3, Okerlund does not teach or suggest the use of decision trees. Claim 22 further 
comprises "recording in a plan domain file each screen of the presentation layer of said computer 
application, the keystrokes necessary to reach each state of each screen of said computer application, 
the states of each screen, and the effect of any actions taken on each screen." As discussed above in 
connection with claim 5, Okerlund does not teach or suggest the use of a "domain file" as is understood 
by one of ordinary skill in the art in light of the claims and specification. 

Examiner's response: 

I1 1 1 . below applies. The claims and only the claims form the metes and bounds 

of the invention. The Examiner has full latitude to interpret each claim in the broadest 

reasonable sense. Limitations appearing in the specification but not recited in the claim 

are not read into the claim. See discussions related to claims 1, 3 and 5 above. 

Applicant is silent regarding "recording in a plan domain file each screen of the 

presentation layer of said computer application, the keystrokes necessary to reach each 

state of each screen of said computer application, the states of each screen, and the 

effect of any actions taken on each screen." First Office Action applies. 



Examination Considerations 

9. The claims and only the claims form the metes and bounds of the invention. 
"Office personnel are to give the claims their broadest reasonable interpretation in light 
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of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55. 44USPQ2d 1023, 
1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited in 
the claim are not read into the claim. In re Prater, 415 F.2d, 1393, 1404-05, 162 USPQ 
541, 550-551 (CCPA 1969)" (MPEP p 2100-8, c 2, 1 45-48; p 2100-9. c 1. 1 1-4). The 
Examiner has full latitude to interpret each claim in the broadest reasonable sense. 
Examiner will reference prior art using terminology familiar to one of ordinary skill in the 
art. Such an approach is broad in concept and can be either explicit or implicit in 
meaning. 

10. Examiner's Notes are provided with the cited references to prior art to assist the 
applicant to better understand the nature of the prior art, application of such prior art 
and. as appropriate, to further indicate other prior art that maybe applied in other office 
actions. Such comments are entirely consistent with the intent and spirit of compact 
prosecution. However, and unless othen/vise stated, the Examiner's Notes are not prior 
art but a link to prior art that one of ordinary skill in the art would find inherently 
appropriate. 

1 1 . Unless otherwise annotated, Examiner's statements are to be interpreted in 
reference to that of one of ordinary skill in the art. Statements made in reference to the 
condition of the disclosure constitute, on the face of it. the basis and such would be 
obvious to one of ordinary skill in the art, establishing thereby an inherent prima facie 
statement. 

12. Examiner's Opinion: Iflj 9-1 1 apply. The Examiner has full latitude to interpret 
each claim in the broadest reasonable sense. 
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Conclusion 

13. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

14. Claims 1-24 are rejected. 



Correspondence Information 

Any inquiry concerning this information or related to the subject disclosure 
should be directed to the Primary Examiner, Joseph P. Hirl, whose telephone number is 
(571 ) 272-3685. The Examiner can be reached on Monday - Thursday from 
6:00 a.m. to 4:30 p.m. 

If attempts to reach the Examiner by telephone are unsuccessful, the 
Examiner's supervisor, David R. Vincent can be reached at (571) 272-3080. 
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Any response to this office action should be mailed to: 

Commissioner of Patents and Trademarks, 

Washington, D. C. 20231; 
Hand delivered to: 

Receptionist, 

Customer Service Window, 
Randolph Building, 
401 Dulany Street, 
Alexandria, Virginia 22313, 

(located on the first floor of the south side of the Randolph Building); 
or faxed to: 

(571) 273-8300 (for formal communications Intended for entry. 
Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.aov . Should 
you have any questions on access to Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll free). 




ifoseph P. Hid 
Primary Examiner 
January 18, 2006 



